CBT  Data  /Knowledge  Acquisition:  U...ge  Objects  to  Prototype  Courseware 


AB-20  -  Paper 

CBT  Data  /Knowledge  Acquisition:  Using  Knowledge  Objects  to  Prototype 

Courseware 

Daniel  J.  Muraida,  Ph.D. 

International  Horizons  Unlimited  San  Antonio,  Texas  78229 

Gary  R.  Grimes,  M.S. 

Metrica,  Inc.  San  Antonio,  Texas 

Saul  B.  Wilen,  M.D. 

Medical  Horizons  Unlimited  San  Antonio,  Texas 


Introduction 

One  of  the  major  factors  underlying  the  considerable  initial  costs  of  CBT 
development  is  the  time  required  of  specialists:  Programmers,  instructional 
designers,  authors,  and  subject  matter  experts  (SMEs).  The  SME  occupies  a  pivotal 
role  in  the  success  or  failure  of  any  instructional  design  project.  Ideally  the  SME 
can  save  the  developers  vital  production  time  when  he  or  she  provides  them 
annotated  or  structured  documentation  that  shows  how  concepts  and  skills  fit 
together  in  a  given  piece  of  curriculum  (Greer,  1991).  Although  knowledge 
acquisition  has  long  been  recognized  as  a  linchpin  process  in  the  development  of 
courseware,  the  process  for  collecting  the  vital  content  data  has  been  based 
primarily  on  unstructured  interviews  and  focus  groups  which  often  result  in 
unclearly  specified  or  misspecified  requirements  (Greer,  1991).  The  following  is  an 
approach  to  knowledge  acquisition  that  will  reduce  the  time  required  of  an  SME, 
while  making  that  time  spent  more  focused  and  useful  for  the  production  process. 

Knowledge  acquisition  as  conceptualized  here  is  based  on  instructional  design 
research  that  has  successfully  used  an  object-oriented  view  of  the  design  process 
(Gonzalez  and  Vavik,  1994;  Merrill,  1997).  The  subject  matter  knowledge  of  an 
SME  is  viewed  as  structured  content  that  can  be  organized  into  a  pedagogical 
structure,  that  will  translate  into  instructional  delivery  decisions.  Knowledge 
acquisition  is  further  viewed  as  an  early  step  in  an  iterative  instructional  design 
process  that  minimizes  risk  for  the  designers  as  well  as  the  client.  We  find  that  a 
courseware  engineering  model  that  emphasizes  rapid  prototyping  (Goodyear,  1994) 
is  most  compatible  with  our  views  of  the  instructional  design  process  as  inherently 
iterative  (Tennyson,  1997). 

Rapid  Prototyping  Approach  to  Courseware  Production 

Until  six  or  eight  years  ago  courseware  was  primarily  a  process  that  required  the 
client(s)  to  endorse  a  design  specification  before  it  could  be  implemented.  Yet 
problems  often  arose  with  this  arrangement  because  the  client  was  often  not  clear 
about  his  or  her  instructional  needs,  or  had  difficulty  in  understanding  the 
developer’s  interpretation  of  those  needs  as  set  forth  in  the  requirements.  Because 
of  the  prohibitively  expensive  results  of  building  something  that  the  client  doesn’t 
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want,  the  rapid  prototyping  approach  emerged.  The  focus  of  the  rapid  prototyping 
approach  is  risk  reduction  .  It  consists  of  a  series  of  iterations  through  a  sequence 
that  includes  specification,  design,  implementation,  and  review,  through  which  the 
client  and  the  developers  make  increasingly  accurate  approximations  to  the  desired 
outcome  (Cooke,  1994;Goodyear,1997). 

Our  approach  to  knowledge  acquisition  is  designed  to  be  a  component  of  a  rapid 
prototyping  process.  More  specifically  we  consider  our  approach  to  be  consistent 
with  Boehm’s  (1988)  spiral  model  of  rapid  prototyping.  This  means  that  the  when 
we  collect  information  on  lesson  subject  matter  from  the  SME  we  are  in  a 
continuous  cycle  through  four  successive  activities  :Risk  assessment, 
implementation,  refinement,  and  integrity  checking.  Viewed  in  the  most  general 
terms  we  begin  by  asking  questions  that  assess  the 

uncertainties  that  surround  clients’  requirements,  and  the  costs  of  making 
implementation  mistakes.  The  initial  implementation  of  the  prototype  focuses  on 
those  aspects  that  have  the  highest  risk  and  cost  of  misspecification..  The  following 
activity  is  refinement,  targeting  the  parts  of  the  prototype  that  still  retain  the  high 
risks  and  costs.  Refinement  focuses  on  making  the  prototype  more  concrete  and 
specific.  The  next  identifiable  stage  is  when  the  prototype  is  refined  to  the  point 
where  it  can  be  checked  for  compliance  with  the  clients’  current  understanding  of 
the  requirements.  Subsequently  the  products  of  this  refinement  process  are  checked 
to  determine  whether  they  are  ready  for  integration. 

Representing  Knowledge  for  Prototype  Instructional  Designs 

Our  view  of  knowledge  representation  is  based  on  the  instructional  design  research 
of  Merrill(1997)  and  Tennyson  (1997).  According  to  the  rapid  prototyping 
approach  to  courseware  development,  the  knowledge  representation  of  the  course 
content  can  be  revisited,  but  for  specific  purposes  dictated  by  a  review  process  that 
minimizes  costs.  The  instructional  design  approaches  of  Merrill  and  Tennyson 
recognize  the  iterative  nature  of  the  instructional  design  cycle,  including  the 
knowledge  acquisition  phase. 

Although  Merrill’s  theorizations  and  models  of  instructional  design  tend  to  be 
abstract,  they  provide  a  highly  flexible  method  of  explaining  how  the  structure  of 
knowledge  can  be  coupled  to  the  structure  of  efficient  and  effective  instruction. 
Borrowing  the  concept  of  an  object  from  artificial  intelligence,  Merrill  defined  the 
concept  of  a  knowledge  object  as  a  set  of  elements,  referred  to  as  a  frame.  The  term 
frame  comes  from  an  analogy  to  a  single  frame  in  a  film  (Minsky,  1974).  The 
concept  is  that  frames  have  an  internal  structure  consisting  of  slots  (that  can  have 
different  values)  and  by  virtue  of  similarities  in  their  internal  structure  they  are 
related  to  other  frames.  Just  as  certain  frames  of  a  film  are  related  because  they 
contain  similar  elements  (e.g.,  location,  themes)  that  could  be  instances  of  a  common 
class  or  subclass,  so  knowledge  frames  can  be  related,  and  more  specifically,  so  can 
content  for  instructional  purposes. 

Merrill  postulates  four  basic  kinds  of  instructional  design  knowledge  frames: 
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Entities,  activities,  processes  and  properties.  Entities  are  essentially  things  that 
qualify  as  nouns.  Activities  are  sets  of  related  actions  that  are  performed  by  a 
learner.  Processes  are  sets  of  related  actions  that  take  place  without  the 
participation  of  the  learner.  Properties  are  qualities  or  quantities  that  are 
associated  with  entities,  activities,  or  processes.  Relationships  among  frames  are 
viewed  as  elaborations  that  comprise  a  frame  network.  The  elaborations  of  this 
network  comprise  the  relationships  among  the  knowledge  objects  (Merrill,  1997). 
Merrill  proposes  three  basic  ways  in  which  those  objects  can  be  related:  They  can 
be  related  via  a  component  relationship,  i.e.,  a  relationship  that  denotes  that 
something  is  part  of  a  larger  structure.  They  can  be  linked  by  an  abstraction 
relationship.  This  means  that  an  entity,  activity  or  process  could  be  viewed  as  a 
subclass  or  superclass  of  other  entities,  activities  or  processes.  They  can  also  be 
linked  by  meaningful  connections  called  associations.  An  example  of  an  association 
link  would  be  the  inclusion  of  information  regarding  the  power  requirements  for  a 
desktop  computer  as  part  of  the  total  information  comprising  a  lesson  on  the  parts 
of  the  computer. 

How  does  this  approach  translate  into  more  efficient  and  cost  effective  knowledge 
acquisition?  Employed  within  a  rapid  prototyping  approach  to  instructional  design 
an  object-oriented  knowledge  acquisition  method  uses  knowledge  frames  or  objects 
and  their  elaborations  as  the  template  for  the  types  of  questions  that  will  be 
presented  to  the  SME.  In  essence,  the  types  of  information  are  to  be  obtained  from 
the  SME  include  The  applications  and  appropriateness  of  this  content  to  the 
specified  student  characteristics;  the  application  of  this  content  to  the  specific 
instructional  environment;  and  the  accuracy  and  clarity  of  this  instructional 
content. 

Current  Use  of  Knowledge  Objects 

At  present  we  employ  a  draft  protocol  to  implement  our  approach.  Data  is  collected 
on  student  characteristics,  instructional  settings,  and  content  structure.  The 
protocol  requests  the  following  applications  of  the  content  be  addressed  by  the  SME 
relating  to  the  audience:  Geographic  location,  age,  need  for  motivation,  skill  levels, 
individual  cognitive  ability  differences,  and  learning  style.  Instructional  setting 
information  consists  of  the  following:  Individual  versus  type  of  group-based 
instruction;  instructor  led,  automated,  or  a  combination  of  both;  distant  versus 
co-located.  Instructional  content  is  captured  through  a  series  of  items  that  uses 
questions  based  on  the  combination  of  knowledge  frames  with  elaboration  types  to 
determine  the  kind  of  knowledge  that  needs  to  be  learned. 

As  An  Example: 

An  opening  query  would  ask  the  following:  How  would  you  categorize  what  you 
plan  to  teach:  (a)  A  device,  object,  creature,  person,  place  or  symbol;  or  (b)  Actions 
to  be  performed  by  a  learner;  or  (c)  Actions  that  do  not  involve  the  learner”. 
Assuming  that  the  SME  responds  that  s(he)  wishes  to  teach  a  process  (option  b), 
such  as  learning  to  drive  a  manual  transmission  vehicle,  the  follow  up  question 
might  take  the  following  form(s):  "What  parts  of  this  process  do  you  wish  to 
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teach?”;  ”Is  this  a  general  process  or  particular  to  one  type  of  vehicle?”. 

Answers  to  the  protocol  provide  the  initial  structure  for  the  instructional  designers 
to  identify  important  learner  and  setting  variables,  instructional  goals,  learning 
objectives,  and  instructional  strategies.  How  does  the  process  of  answering  a 
sequence  of  questions  provide  all  that  information?  It  does  so  by  virtue  of  the 
knowledge  frames.  If  the  draft  protocol  calls  for  all  the  data  mentioned  above,  then 
an  example  of  the  answers  to  just  that  short  query  would  be  as  follows: 

Student  Characteristics 


Geographic  Location:  Southwestern  US;  Inner  city  school  district 
Age:  15-17 

Motivational  Level:  High  (Students  require  little  encouragement) 

Skill  Level:  Low  (Fewer  than  20%  have  experience  with  manual  transmission) 

Individual  Cognitive  Ability  Differences  (Minor;  Standardized  scores  generally 
Homogeneous) 

Learning  Style(External  control) 

Instructional  Setting 
Individual  instruction 
Automated 
Co-located  instruction 
Instructional  Content 

Knowledge  Frame:  Activity  -  Actions  to  be  performed  by  a  learner 

Elaboration:  An  abstraction  relationship  -  General  method  of  using  a  stick  shift 
transmission  in  sedans 


What  have  we  gleaned  at  this  point?  The  students  are  from  an  inner  city  learning 
environment,  with  little  prior  manual  transmission  experience,  but  they  are 
motivated  to  learn  the  skill.  Moreover,  they  are  generally  homogeneous  in  terms  of 
cognitive  ability,  and  they  learn  more  readily  with  directive  instruction.  With 
regard  to  the  planned  instructional  setting,  it  is  expected  that  the  students  will  learn 
the  use  of  a  manual  transmission  individually,  from  courseware.  This  information  is 
rather  easy  to  obtain,  given  that  the  protocol  provides  explanations  of  technical 


4 


CBT  Data  /Knowledge  Acquisition:  U...ge  Objects  to  Prototype  Courseware 


terms  such  as  cognitive  ability 

and  learning  style.  What  is  difficult  to  obtain  is  the  content  structure  and  the 
potential  links  to  instructional  strategies  and  delivery  methods  (media),  but  that  is 
exactly  the  kind  of  information  that  a  knowledge  object  based  protocol  elicits. 
Given  that  the  content  to  be  learned  is  an  activity,  and  it  is  a  generic  activity, 
appropriate  instructional  strategies  can  be  identified  through  the  application  of 
instructional  models  that  address  the  identified  learning  requirements.  In  this 
example  the  generic  activity  could  be  taught  by  a  series  of  interactions  between 
student  and  instructional  system  known  as  an  execution  transaction  (Merrill,  1997) 
or  as  an  intellectual  (procedural)  skill  (Gagne,  1993),  just  to  name  two  of  the 
available  instructional  strategies.  This  technique  is  in  development.  Structured 
interviews  are  the  first  step  for  automated  knowledge  elicitation  processes, 
especially  if  combined  with  other  techniques(Cooke,  1994;  Shute,  Torreano,  & 
Willis,  1998).  The  major  problem  in  automating  a  knowledge  elicitation  method  for 
use  with  courseware  authoring  systems  lies  in  the  interpretability  of  the  data  the 
authoring  system  receives.  Recent  attempts  to  automate  knowledge  elicitation 
techniques  have  proved  encouraging  (Shute,  et  al ,  1998),  but  lacking  is  a  language 
that  can  address  the  many  different  kinds  of  knowledge  types  while  linking  them  to 
their  corresponding  instructional  and  delivery  strategies,  irrespective  of  authoring 
language.  It  is  our  contention  that  knowledge  objects  provide  the  bridge  to 
authoring  the  appropriate  strategies  because  they  elicit  content  knowledge  in  terms 
that  can  be  manipulated  in  most  existing  authoring  languages.  These  instructional 
strategies  can  be  organized,  selected,  and  implemented  by  couching  them  in  an 
object-oriented  framework.  Rapid  prototyping  facilitates  the  transmission  of  high 
quality  data  between  the  SME  and  the  designers.  Data  quality  will  be  enhanced  by 
more  accurate  initial  assessments  and  implementations,  and  refinements,  resulting 
in  CBT  that  embodies  the  instructional  environment  that  meets  the  standards  for 
both  clients  and  designers.  Ongoing  evaluations  that  examine  the  versions  of  this 
object  oriented  approach  to  instructional  knowledge  acquisition  will  define  long 
term  applications  for  CBT  development. 
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